On the residual finiteness of outer automorphisms 
of relatively hyperbolic groups 

V. Metaftsis and M. Sykiotis 
Abstract 

We show that every virtually torsion-free subgroup of the outer au- 
tomorphism group of a conjugacy separable hyperbolic group is resid- 
ually finite. As a result, we are able to prove that the group of outer 
automorphisms of every finitely generated Fuchsian group and of ev- 
ery free-by-finite group is residually finite. In an addendum, we also 
generalise the main result for relatively hyperbolic groups. 

1 Introduction 

The mapping class group of a compact connected, possibly with boundary, 
surface S, Mods, is defined as the group of isotopy classes of homeomor- 
phisms S —* S. The algebraic investigation of that group goes as far back 
as Dehn and Nielsen, who first showed that the mapping class group of an 
orientable surface is embedded into the group of outer automorphisms of 
the fundamental group of S, Out(7Ti(5)). This embedding was later ex- 
tended to a wider class of 2-orbifolds. Maclachlan and Harvey ^5] proved 
that Mods < Out(7Ti(S)) for a surface S obtained from a compact surface 
of genus k by deleting s points, t discs and r marked points. Recently, Fuji- 
wara extended the above embedding to the case of hyperbolic 2-orbifolds 
(orientable or non-orientable) with finite volume. 

Grossman |12| was the first to show that if Sk is a compact orientable 
surface of genus k, then Mods k is residually finite. Recently, Allenby, Kim 
and Tang Tj extended the result of Grossman for non-orientable closed sur- 
faces. Ivanov Jl] gave a geometric proof of the result of Grossman which 
seems that can easily be extended to the non-orientable case. The proofs 
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of both results in ^2] an d CP are using combinatorial group theory argu- 
ments. One of the key ingredients of both results is the fact that the group 
of conjugating automorphisms of the fundamental group of the surface co- 
incides with the group of its inner automorphisms. If G is a group, then 
an automorphism / of G is called a conjugating automorphism if f(g) is a 
conjugate of g for every g € G. The conjugating automorphisms of a group 
G form a subgroup of the group of automorphisms of G, Aut(G), which we 
denote by Conj(G). Moreover, Conj(G) is normal in Aut(G) and contains 
the subgroup of inner automorphisms of G, Inn(G). 

In the present note we investigate the residual finiteness of Out(G) for 
hyperbolic groups G. Our study is, as well, based on the investigation of 
the group of conjugating automorphisms of G. In fact, using the powerful 
geometric ideas developed by Paulin [201 an d Bestvina [H] in the context 
of ultralimits of metric spaces, we show that in a hyperbolic group G, the 
quotient group Conj(G)/Inn(G) is always finite. The main theorem shows 
that every virtually torsion-free subgroup of the outer automorphism group 
of a conjugacy separable, hyperbolic group is residually finite. 

As a result, we obtain the residual finiteness of the outer automorphism 
group of finitely generated Fuchsian groups and of free- by- finite groups. 
Consequently the mapping class groups of hyperbolic 2-orbifolds with finite 
volume are residually finite. Hence, we retrieve the results of Grossman and 
of Allenby, Kim and Tang as special cases of our corollaries. 

In an addendum at the end of the paper we show how to generalize the 
main results for relatively hyperbolic groups. 

2 Main Results 

The next lemma is what really proved in j!21 Theorem 1]. We reproduce 
here the proof for the reader's convenience. 

Lemma 2.1. Let G be a finitely generated, conjugacy separable group. Then 
the quotient group Aut(G)/Conj(G) is residually finite. 

Proof. Since G is finitely generated, we can find a family (iTj)jg/ of charac- 
teristics subgroups of finite index in G such that for each normal subgroup 
N of finite index in G there is a subgroup Ki in the above family, contained 
in N. Every automorphism f of G induces an automorphism fi of G/Ki 
which acts as a permutation on the conjugacy classes of G/Ki. Therefore, 
for each i 6 I we have a homomorphism 7Tj : Aut(G) — > Si, where Si denotes 
the permutation group on the set of conjugacy classes of the finite group 
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G/Ki. Since each conjugating automorphism preserves conjugacy classes, 
the group Conj((x) is contained in the intersection Hiei" Ker(7Tj) of the ker- 
nels of the 7Tj's. Now let / G Hie/ Ker(7Tj). Then for every g E G, the 
elements f(g) and g are conjugate in each quotient G/Ki. In particular, the 
elements f(g) and g are conjugate in each finite quotient of G and thus they 
are conjugate in G, by the conjugacy separability of G. This means that / is 
a conjugating automorphism of G and hence Conj(G) = C\ ieI Ker(7Tj). □ 

The following material on ultralimits of metric spaces can be found in 
detail in the paper of Kapovich and Leeb [To] . 

A filter on the set of natural numbers N is a non-empty set u of subsets 
of N with the following properties: 

1. The empty set is not contained in u. 

2. u; is closed under finite intersections. 

3. Ifiew and AC B, then B G oj. 

A filter u is called ultrafilter if it is maximal. An ultrafilter is called 
non-principal if it contains the complements of the finite subsets of N. Let 
u be an ultrafilter on N and a : N — > [0, co) a sequence of non-negative real 
numbers. Then there is a unique point I, which we denote by lim^aj, in the 
one-point compactification [0, oo] of [0, oo) such that for each neighborhood 
U of I, the inverse image a~ l {U) of U under a is contained in u. 

Let (Xi,di,Xi)izN be a sequence of based metric spaces and let to be 
an ultrafilter on N. On the subspace Y of IlieN^ consisting of all se- 
quences (x{) for which lim^d^Xj, x°) < oo, we define a pseudo-metric d w by 
duj((xi),(yi)) = limu (di(xi,yi)). The based ultralimit (A" w ,d w ,x w ), where 
Xuj = (^i)ieN 5 is the associated metric space. 

Let X be a geodesic metric space and 5 a non-negative real number. 
The space X is called S -hyperbolic if for every triangle A C X with geodesic 
sides, each side is contained in the ^-neighbourhood of the union of the two 
other sides. 

Let G be a finitely generated group and let X = X(G, S) be the Cayley 
graph of G with respect to a finite generating set S for G, closed under 
inverses. The group G is called 5-hyperbolic if X is a 5-hyperbolic space 
with respect to the word metric. 

In the next lemma we use various results concerning actions of groups on 
HL-trees. For details, we refer the reader to the paper of Morgan and Shalen 

ESI- 
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The key point of Lemma 12.21 is the following statement. Assume that 
an action of a hyperbolic group G on a real tree Y is obtained as a limit 
of a sequence of actions of G on its Cayley graph X, where each action in 
the sequence is the natural action of G on X twisted by an automorphism 
of G. If the conjugacy class of an element g in G is periodic under these 
automorphisms, then g is elliptic when acting on the limit tree Y. 

As it was pointed out by the referee, the above statement seems to be 
well known to the experts (see 0]). Nonetheless, the authors failed to track 
down a reference for its proof. So, a proof of it, is included in the lemma for 
the reader's convenience and completeness. 

Lemma 2.2. Let G be a hyperbolic group. Then the group Inn(G) of inner 
automorphisms of G is of finite index in Conj(G). 

Proof. Suppose to the contrary that Inn(G) is of infinite index in Conj(G). 
Fix an infinite sequence fx, /2, ...,/„,.. . of conjugating automorphisms of 
G representing pairwise distinct cosets of Inn(G) in Conj(G). We apply the 
method of Bestvina and Paulin, using ultrafilters, to construct an M-tree on 
which G acts by isometries. 

Let X = X(G,S) be the Cayley graph of G with respect to a finite 
generating set S closed under inverses. Then X with the associated word 
metric d is a <5-hyperbolic metric space for some 5 > 0. Each /, gives 
an action pi by isometries of G on X by pi(g,x) = fi(g)x. The outer 
automorphism group of a virtually cyclic group is finite. Thus, we may 
assume that G is non-elementary. In that case the action of G on the 
boundary dX of X is non-trivial and Lemma 2.1 in |2L)| applies to any 
action pi , yielding a sequence x\ of elements of X such that 

max d{x° i ,fi{g)xf) < maxd(x, fi(g)x) (1) 
for all x in X. Let Aj = max d( x^ , fi(g)x ( -). Then 

<>*\\g\\ (2) 

for all g in G, where \\g\\ denotes the word-length of g with respect to S. 
If the sequence (Aj) contains a bounded subsequence, then the argument in 
[201 Case 1, p. 338] shows that there are indices i and j with i ^ j such 
that the automorphisms fi and fj differ by an inner automorphism of G 
and consequently they give rise to the same coset of Inn(G), contradicting 
the choice of the fi. It follows that lim Aj = oo. We consider the sequence 
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(Xi, di,x®) of based metric spaces, where X{ = X and d% = j> Note that the 
space (Xi,di,x®) is ^--hyperbolic for all i. For any non-principal ultrafilter 
ijj on N, let (X u , d^, xj) be the corresponding based ultralimit. The fact 
that the distance d 0J (x,y) of two points x = (x^ and y = (yi) of X^ is 
approximated by the distances di(xi,yi) for infinitely many i, implies that 
X^ is a 0- hyperbolic space (i.e., an R-tree), since lim j- = 0. The action of 



i— >oo 



G on X u is given by g ■ (x^) = (fi(g)xi). Inequality (J2J) ensures that the 
action is well-defined. We will show that the action of G on X w is trivial 
(i.e. there is a global fixed point). 

Let g be an element of G which acts as a hyperbolic isometry on X u , 
and let T w (g) denote its translation length. Fix x = (xj) € X w such that x 
lies on the axis of g. Then 

Tu(g) = du{gx,x) = Qhawditfi^x^Xi) =) du - (<> 1 



n 



for all positive integers n. In particular, d UJ (gx,x) = —^-z ' — ~ an d thus 

]hn w (2d i (fi(g)xi,Xi) - di(fi(g) 2 Xi,Xi)^ =0. (3) 

For each i, we fix an element yi of X on which the displacement func- 
tion of fi(g) attains its minimum T(fi(g)), i.e. r(fi(g)) = d(fi(g)yi,yi) = 
M{d(fi(g)y,y)\y £ X}. 1 Since X is a 5-hyperbolic space, there is a non- 
negative constant K(5) depending only on 5 such that 



d(fi(g)xi,Xi) > 2d(x t ,y i )+r(f i (g))-K(5). (4) 



Then, 



A = 2d(fi(g)xi,Xi) — d(fi(g) 2 Xi,Xi) 

> Ad(xi,yi) + 2r(/i(s)) - 2K(S) - d{f % (g) 2 Xil x<) 

> 4d(xi, Vi ) + 2r(fi(g)) - 2K{5) - 2d{x hyi ) - 2r(fi(g)) { °> 
= 2d(x i ,y i )-2K(5), 

where the second inequality follows from the triangle one. Working in a 
similar way, we see that 

2d( Xi , yi ) - K{5) < d{ji{g)x u xi) - r(/ifo)) < 2d(x u Vi ) (6) 



1 The reader should not confuse this with the algebraic translation length of the elements 
of a group G. 
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and hence we can say that 



\d{fi(g)x uXl ) -r(fi(g))\ < 2d(x u yt) + K{5). 



(7) 



Consequently, 



< 




< 


Tu>(g) 


< 


Tu{g) 



d t (h(g)x l ,x t )- T -^l 



+ 



+ 2 d(xj,yj) K{5) 
A* A^ 

+ \A\ +3 m i 



where the second inequality follows from Q and the third one from (jSJ). 
The w-limit of each term in the right-hand side of the above inequality is 
(for the second term see ©). Therefore 



Tu(g) = lim, 



r{h{g)) 
' Xi 



0. 



since r(fi(g)) = r(g) for all i (fi being a conjugating automorphism). Hence, 
each element g of the finitely generated group G acts as an elliptic isometry 
on X u . This means that the action of G on X w has a global fixed point, 
say z = (zi). It follows that for every e > and every finite subset F of G, 
there is an Q E u> such that 



di{zu fi{g)zi) 



d{ziji{g)zj) 



< e, 



for all z G and g € F. This contradicts to the minimality of A, (see 
©)• □ 

We should mention here that this is the best possible result in that gen- 
erality. Indeed, as shown by Burnside an d subsequently by several other 
authors (see also Sah |25)) there are finite groups that posses non-trivial 
outer conjugating automorphisms (known also as outer, class preserving au- 
tomorphisms). Therefore one can easily construct free-by-finite groups with 
outer conjugating automorphisms by considering the direct product of the 
above mentioned finite groups by free groups. 

We are now able to show our main theorem. 

Theorem 2.3. Let G be a conjugacy separable, hyperbolic group. Then each 
virtually torsion-free subgroup of the outer automorphism group Out(G) of 
G is residually finite. 
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Proof. It suffices to show that each torsion-free subgroup H of Out(G) is 
residually finite. We consider the following short exact sequence 

1 Conj(G)/lnn(G) -U Aut(G)/lnn(G) A Aut(G)/Conj(G) ->• 1 . 

By Lemma 12.21 the first term is finite. This implies that the restriction 
of 7r on H is a monomorphism. It follows that H is residually finite being 
isomorphic to a subgroup of Aut(G)/Conj(G), which is residually finite by 
Lemma 12.11 □ 

In view of the above theorem, given a group G it is natural to seek 
conditions under which the outer automorphism group Out(G) of G is vir- 
tually torsion- free. Recently, Guirardel and Levitt JB] have shown that the 
outer automorphism group of a hyperbolic group G is virtually torsion- free, 
provided that G is virtually torsion-free. Thus, by Theorem 12.31 the outer 
automorphism group of a conjugacy separable hyperbolic group is residually 
finite, since a residually finite hyperbolic group is virtually torsion-free. 

The next lemma is more or less known (see |131 117j ). 

Lemma 2.4. Let G be a finitely generated group containing a normal sub- 
group N of finite index whose center is trivial. IfOut(N) is virtually torsion- 
free, then so is Out(G). 

Proof. Let AutAr(G) denote the subgroup of Aut(G) consisting of those au- 
tomorphisms of G which fix TV" and induce the identity on G/N. The re- 
striction map <p : AutAr(G) — > Aut(iV) is an injection. Indeed, let g G G 
and / € Autjv(G). Then f(g) = gh g for some h g G N. Suppose now 
that / is in the kernel of the restriction map. Then for each h in N we have 
ghg^ 1 = fighg^ 1 ) = gh g hh g 1 g~ 1 . This implies that h g is in the center of N, 
which is trivial, and therefore / is the identity. From the injectivity of <f> we 
get ^(innjv(G)) = Inn(iV), where Inn/v(G0 denotes the (normal) subgroup 
of Inn(G) consisting of all inner automorphisms of G induced by elements 
of N. We conclude that the quotient group Aut7v(G)/InnAr(G) embeds into 
Out(iV). In particular, Autjv(G)/Inn7v(G) is virtually torsion- free. 

Now the restriction of the natural projection ir : Aut(G) — > Out(G) to 
AutAr(G) induces a map tt : AutAr(G)/Inn7v(G') — > Out(G). The kernel of fr 
(which is equal to Inn(G)/Innjv(G ! )) is finite, since ./V is of finite index in G, 
while its image has finite index in Out(G), since AutTv(G) is of finite index 
in Aut(G), by |17l Lemma 1]. It follows that each finite-index, torsion- 
free subgroup of AutAr(G)/Inn7v(G) embeds as a subgroup of finite index in 
Out(G), which proves the lemma. □ 
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Corollary 2.5. The outer automorphism group of a finitely generated, free- 
by-finite group is residually finite. 

Proof. It is well-known that every finitely generated, free-by-finite group G 
is hyperbolic. Furthermore, G is conjugacy separable by |S]. Therefore 
Theorem 12.31 applies to G. On the other hand, it is also known that the 
outer automorphism group of a free group is virtually torsion-free. If G is 
virtually infinite cyclic, then Out(G) is finite. In the case where G is not 
virtually cyclic, the center of a free subgroup of finite index in G is trivial, 
and the result follows from Lemma 12.41 □ 

Recall that a Fuchsian group is a discrete subgroup of the group of 
isometries of the hyperbolic plane H 2 . 

Corollary 2.6. The outer automorphism group of a finitely generated, Fuch- 
sian group is residually finite. 

Proof. Every finitely generated Fuchsian group G contains a normal torsion- 
free subgroup of finite index, say N, which is either a free group or the 
fundamental group of an orientable surface group of genus g > 2. So N is 
hyperbolic since it is either free or quasi-isometric to H 2 . Moreover N is 
conjugacy separable |22j Theorem 3.3] and its outer automorphism group 
Out(iV) is virtually torsion- free and so N satisfies the hypotheses of Theorem 
12.31 Hence, Out(iV) is residually finite. 

If N is cyclic then G is virtually cyclic and so Out(G) is finite. In 
all other cases, N is a normal subgroup of finite index in G with trivial 
centre (since it is a non-cyclic torsion-free hyperbolic group) and so from 
the proof of Lemma f2. 41 we have that AutAr(G)/Inn7v(G) is a subgroup of 
Out (AT"). Hence, every subgroup of Aut7v(G)/InnAr(G) is residually finite. 
But, again from the proof of Lemma 12.41 there is a torsion-free subgroup 
of AutAr(G)/Inn7v(G) that embeds as a finite index subgroup in Out(G). 
Therefore, Out(G) is residually finite. □ 

The corollary below generalizes the results of Grossman ^2] and of Al- 
lenby, Kim and Tang 1 . Notice that for the exceptional cases of the Torus 
and the Klein bottle it is easily verified that the result still holds. 

Corollary 2.7. The mapping class group Mods of a hyperbolic 2-orbifold S 
with finite volume is residually finite. 

Proof. The fundamental group of a hyperbolic 2-orbifold with finite volume 
is a finitely generated Fuchsian group. Hence, the proof is an immediate 
consequence of the results of Fujiwara JO]) the above corollaries and the 
fact that subgroups of residually finite groups are residually finite. □ 
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Addendum. Added, October 8, 2005. 

Only recently the authors found out that their main result can be generalized 
to relatively hyperbolic groups by using [2j Theorem 1.1]. 

Relatively hyperbolic groups were introduced by Gromov in in or- 
der to generalize notions such as the fundamental group of a complete, non- 
compact, finite volume hyperbolic manifold and to give a hyperbolic version 
of small cancellation theory over free groups by adopting the geometric lan- 
guage of manifolds with cusps. 

This notion has been developed by several authors and, in particular, 
various characterizations of relatively hyperbolic groups have been given 
(see EH and [7] and references therein) . 

We recall here one of Bowditch's equivalent definitions. A finitely gener- 
ated group G is hyperbolic relative to a family of finitely generated subgroups 
Q if G admits a proper, discontinuous and isometric action on a proper, hy- 
perbolic path metric space X such that G acts on the ideal boundary of X 
as a geometrically finite convergence group and the elements of Q are the 
maximal parabolic subgroups of G. 

We should mention here that Farb [S] introduced a weaker notion of 
relative hyperbolicity for groups using constructions on the Cayley graph of 
the groups. 

Except of the fundamental groups of hyperbolic manifolds of finite vol- 
ume, another interesting family of relatively hyperbolic groups are the funda- 
mental groups of graphs of finitely generated groups with finite edge groups 
which are hyperbolic relative to the family of vertex groups (since their 
action on the Bass-Serre tree satisfies definition 2 in jSJ). 

The key lemma of the paper, Lemma 2.2, generalizes to relatively hy- 
perbolic groups. 

Lemma 2.2/ Let G be a relatively hyperbolic group. Then the group Inn(G) 
of inner automorphisms of G is of finite index in Conj(G). 

Sketch of proof. In this case the Cayley graph of G is replaced by the <5- 
hyperbolic metric space X on which G acts by isometries. 

Let Xi = inf max d(x, fi(s)x) where S is a fixed finite generating set G 

and let X® € X such that maxd(x°, fi(s)x ( -) < Xi + \. 

sdS 

By the proof of Theorem 1.1 in [2], the sequence Aj converges to infin- 
ity. Hence for every non-primitive ultrafilter u on N the based ultralimit 
(X u , du, x 1 ^) of the sequence of based metric spaces (X, jr^x®) is an R-tree. 
Moreover, there is an induced non-trivial isometric G-action on (X^, d^, x^ ), 
given by g-Xi = fi(g)xi. 
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Following step- by-step the proof of Lemma 2.2 we arrive again at the 
contradiction that the action has a global fixed point. 
The reader should be careful in the following. 

1. The elements yi of X are chosen such that 

r{fi{g)) < d(/i(s)lfc,l«) < r(fi(g)) + -. 

i 

2. The inequalities (5), (6) and (7) become 



A > 2d(xi,yi) - 2K{5) 



(5') 



2d{x i ,y i ) - K(5) < d(fi(g)xi,Xi) - T(fi(g)) < 2d( Xi , yi ) + - (6') 



and \d(f l (g)x i ,x l )-T(f i (g))\<2d(x l ,y l ) + K(5) + -, (T) 
respectively. 
3. Finally, the last inequality becomes 
r(fi(9)) 



A; 



, / x , / m 1^1 K(S) 3 

< K{9) -di(fi(g)x i ,x i )\ + V +3 ^^ + T V" 



□ 

Consequently, Theorem 2.3 generalizes as follows. 

Theorem 2.3/ Let G be a conjugacy separable, relatively hyperbolic group. 
Then each virtually torsion-free subgroup of the outer automorphism group 
Out(G) of G is residually finite. 
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